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Smoke Gets in Your Eyes: 

Using Rubrics as a Tool for building Justice into Assessment Practices 

Colette R Alexander - 
Christian Heritage College 
calexander@chc.edu.au 

Sadie Praeger 
Christian Heritage College 
spraeger@chc . edu.au 

Abstract 

Students are empowered or disempowered by the information that we provide or withhold from 
them; nowhere is this more significant that in assessment practices (Popham 1997). While some 
students are able to successfully navigate the nuances of assessment, others see assessment as a 
bamboozling array of ‘smoke and mirrors’ designed to shroud achievement in mystery (Martin- 
Kniep 2000; Weiner 2000). In 2008 the academic staff of the School of Education and Humanities 
at Christian Heritage College, a private higher-education provider in South-East Queensland, 
initiated an action research project designed to address the gaps and silences evident in assessment 
processes in order to encourage justice in and equity of engagement with assessment practices for 
all students. The aims of this research were to; 

• Identify aspects of assessment practices that were not being overtly and openly 
communicated with students; 

• Consider the use of rubrics as an assessment tool to address these gaps, silences and 
inconsistencies between the declared and actual assessment processes; 

• Understand how students engage with rubrics as an assessment tool; and 

• Explore the potentialities of using rubrics as a means of leading students towards 
empowered understanding of assessment practices. 

To date, two cycles of action research have been completed (Ereebody 2003; Kemmis & McTaggart 
1988). The first involved a working party of three full-time lecturing staff; the purpose being to 
establish the scope of potential action through the undertaking of an audit of practice, a review of 
literature and the development of recommendations that would structure further action. The second 
cycle involved five full-time staff in applying the nine recommendations with undergraduate 
students across all four years of the Bachelor of Education programs. Data was collected from 
participating lecturers and students across Years 2-4 by the means of email-back questionnaires. 
Lecturers were asked open-ended questions and the data was analysed by the identification and 
classification of unique meaning units leading to emergent categories (Glesne 1999; Johnson & 
Christensen 2004). Students were asked to respond to a yes/no survey questioning the nature of 
their engagement with the rubrics across six possible processes. This data was collated 
quantitatively then linked to the students’ results for each task. Descriptive statistics such as 
frequency distributions and measures of central tendency were used to identify patterns in the data 
relating to rubric usage and corresponding assessment outcomes (Johnson & Christensen 2004). 

The data collected demonstrated that there were differential levels of engagement with rubrics 
within both the staff and student groups. Eor students, the data showed that while not using the 
provided rubrics did not automatically lead to lower levels of achievement, a greater level of 
engagement with the rubrics did correlate with higher levels of achievement. Eor lecturing staff, the 
level of engagement in the use of the recommendations was shown to positively affect the lecturer’s 
impression of the quality of their linguistic clarity in and alignment of teaching and learning with 
assessment practices. 
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The findings from the student data shows that there are, broadly speaking, three types of students in 
relation to assessment praetiees; the mystified, the intuitive and the empowered. Analysis of the 
data eolleeted from both staff and students also supports the eonelusion that rubrie usage does affeet 
learning and assessment outeomes, and that student knowledge of and engagement with rubries will 
positively impaet results. A proeess of learning was also evident in the differential proeesses and 
praetiees of leeturing staff as they engaged with ereating, teaehing about and teaehing with rubries. 
These findings have led to the initiation of a third eyele of aetion designed to engage with more 
leeturing staff, and to investigate what intuitive and empowered students aetually do, in order to 
bring justiee to the assessment praetiees of the mystified. 

Introduction 

Christian Heritage College (CHC) is a small private higher edueation institution offering 
undergraduate and postgraduate eourses aeross four different sehools; ineluding the Sehool of 
Edueation and Humanities (SEH). Over its 23 years of history, the SEH and the College has 
developed a profile and role within the Queensland eontext in teaeher edueation, partieularly in 
relation to the needs of Christian students and sehools. While the College ean be seen to be serving 
a niehe purpose and elientele, the student body at CHC is diverse and a wide speetrum of 
denominational, edueational and soeio-eoonomie baekgrounds is represented in the student 
population. These students report many different reasons for seleeting the College. However, a 
large proportion report seleeting CHC beeause of its Christian foeus and ethos, smaller elass sizes 
and personalised attention. Eaeh year, a signifieant proportion of the student body is made up of 
students who are in need of speeialised assistanee in order to faeilitate their engagement in the 
praetiees of aeademie diseourses (Hyland, 2009). 

As a result, the College has established a range of proeesses and interventions that are used to build 
bridges between where students are at and where they need to be in terms of the proeedures and 
praetiees of an aeademie learning environment. These proeesses and interventions have ineluded 
aetivities sueh as standardised testing and the reporting of strengths and weaknesses to individuals, 
the provision of additional and speeialised tutorials and workshops, personalised eoaehing and 
mentoring, faeilitating self-regulated ‘study-help’ groups, tutoring and maximising opportunities to 
submit, review and resubmit assessment. Over time these aetions have been shown to be 
foundational to establishing the sueeess of students in their first year at CHC. Eor most students 
these types of experienees in their first year at the College establishes a pattern of soeial interaetion 
and engagement in the learning eommunity that beeomes self regulated and the relianee on the 
support praetiees wanes over subsequent semesters. 

There are, however, some students who earry their diffieulties and problems into subsequent years. 
Often, these students have attained aeademie results that are disparate, and their performanee in 
various assessment tasks and units are variable and unpredietable. Yet these students will usually 
also have a keen desire to sueeeed and to find ways to engage in aeademie learning. Ruth is one 
sueh student. 

Ruth is a keen young student who is enrolled in the Baehelor of Arts/Baehelor of 
Eduoation(Seeondary/MiddleYears) program, she hopes to beeome a high sehool Drama and 
English teaeher. Ruth is now in her early twenties and entered College as a sehool leaver. She is 
eurrently in her fourth year of study. While Ruth started well in her first semester, gaining a grade 
point average of 5.4, however, her performanee sinee this time has been variable with her worst 
semester being resulting in a grade point average of just 3.4. Over her program so far, Ruth’s 
results have ineluded two ‘High Distinetion’ and six ‘EaiT and an even spread of everything in 
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between. While there have been a range of personal issues that have impacted upon Ruth’s 
performance across different semesters, it has been the enormous variation of results both within 
and between semesters that has been a point of concern. While some lecturers describe Ruth as an 
engaging and academically capable student, others have expressed a concern that she appears to be 
‘functionally illiterate’ in relation to the specific assessment tasks she has completed for their units. 

Ruth and other students like her have played an important role in the development of this 
investigation into assessment practices. Their pattern of variable engagement and achievement has 
demonstrated the presence of a gap or miscommunication in relation to the expectations and 
requirements of the academic discourses used in assessment. As such, finding ways to engage these 
types of students effectively in academic discourses without compromising academic rigour became 
the focus of professional conversations amongst the faculty. Concerns were raised about the 
‘smoke and mirrors’ that were evident in the ways that both individual lecturers and the faculty as a 
community approached assessment practices. In particular, issues of equity and justice were 
discussed in relation to the impact of the variation and divergence evident in the communication of 
assessment requirements and expectations. From these conversations, the use of rubrics as a tool 
for facilitating the clear and explicit declaration of assessment requirements and expectations was 
identified as a potential means by which just and equitable communication might occur. It was 
hoped that rubrics would assist students like Ruth to unravel the mysteries of assessment and 
achievement and develop some consistency in their performances across a wider range of tasks and 
discourses. 

Investigating the Nature, Role and Purposes of Rubrics 

he use of the term ‘rubric’ as a noun began in the middle ages when it was used to 
describe various uses of red ink, such as the large elaborate red letters that monks 
created to differentiate each new section of a text (Popham 1997, T1 8). This word 
usage originated from the word ‘ruber’ which signified the red earth, and the original 
meaning was to ‘adorn with red’ or to be ‘marked with red’. The use of the term in 
educational contexts began as a way to refer to the ‘red’ notes that teachers make on 
a student’s work, though in recent years its use has been narrowed to represent a very specific type 
of scoring tool (Wikipedia 2008, ^ 1 1). 

A rubric is a teaching tool designed to clearly describe the characteristics and qualities of a specific 
learning or assessment task. They are most commonly used as a means of assessing the quality of 
students’ work, though they can also be used as an instructional tool in helping students to 
understand the requirements of the task (Andrade 2000, p. 13). Montgomery (2000, p. 325) defines 
them as, ‘an assessment tool that uses clearly specified evaluation criteria and proficiency levels to 
gauge student achievement of those criteria.’ Rubrics are always distinguished from other tools 
used for assessing student work by the provision of actual descriptions of the desirable qualities or 
standards of work for the set criteria. 

As such, the foundational purpose of a rubric as a scoring tool is to facilitate understandings of the 
requirements of and guide judgements made about the quality of particular tasks. As Lund (2006, 
p.l3) states, ‘good rubrics should paint verbal pictures of program expectations.’ Montgomery 
(2002) notes that rubrics are being used as scoring tools to; 

• Help instructors define the precise criteria and standards for success in the assessment task 

prior to the completion of the task, 

• Establish criteria prior to instruction so that teachers can focus on critical elements of the 

curriculum thus ensuring alignment of curriculum content, instruction and assessment, 
and 
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• Give feedback to students in such a way that they may achieve success on similar tasks in 

the future. 

Rubrics should be designed to assist students in the process of creating assessment demonstrations 
and teachers in the process of providing informative and timely feedback. 

While rubrics were originally designed to facilitate assessment processes and purposes, their 
potential for facilitating instructional purposes has become the core of much of the recent literature 
(Andrade 2005; Lund 2006; Montgomery 2002). The instructional purposes of a rubric relate to 
their use with students as a means of improving performance. The instructional purposes of rubrics 
are facilitated by activities such as; providing and explaining the rubrics to students when tasks are 
being explained, having students use the rubrics to assess examples of various qualities, co-creating 
rubrics with students, and requiring students to use the rubrics to self-assess their work (Andrade 
2005, p. 29). Rubrics help teachers to unpack assessment processes and products in order to 
scaffold student understanding of and achievement in a particular task. In this way, rubrics can 
become a foundational asset in the pedagogical processes of modelled and guided practice, and 
have the potential to make a positive contribution to students’ transformed practice. There is 
significant evidence that, ‘high levels of rubric use lead to increased academic performance scores’ 
(Loveland 2005, p. 22), though this increase is also dependent upon the quality of the rubrics being 
used. As Popham (1997, T1 36) states, ‘appropriately designed rubrics can make an enormous 
contribution to instructional quality’. 

The benefits of using rubrics are multi-faceted and closely aligned to the purposes for which they 
are being used. That is, there are benefits for both teachers and students in using rubrics as an 
assessment tool, but, these benefits are added to when rubrics are also used as instructional tools. 
The overt benefits of developing assessment and instructional uses for rubrics include the provision 
of: 

• Clear guidelines and expectations for each of the passing grades (Loveland 2005, p. 22), 

• Resources to facilitate self assessment leading to self-correction prior to submission 
(Andrade 2008), 

• A means to identify areas for improvement (Montgomery 2002, p. 36), and 

• Cogent feedback that explains and justifies the grade received (Andrade 2005, p. 29). 

However, a review of the literature has shown that the most powerful benefit of rubric use for 
students relates to the shift in the locus of control over assessment. Weiner’s attributional approach 
to motivation identifies and describes the intersection of the attribution of success or failure over the 
issues of the ‘locus of control’, the ‘stability of causes’ and the ‘controllability of causes’ 
(Lefrancois 2000, p. 427; Weiner 2000, p. 4). These three factors intersect in a ‘three-dimensional 
causal space’ within which the causes of success or failure attributed by an individual can be 
mapped (Weiner 2000, p. 5). The use of rubrics in teaching and learning has the potential to 
positively influence students’ attributions of success or failure by removing the ‘mystery’ of how 
assessment is graded and providing clear directions for improving performance that are within the 
students’ control (Martin-Kniep 2000, p. 41). The ‘locus of control’ of success or failure is shifted 
away from the teacher, who controls and make ‘hidden’ calculations about grades, to the students, 
who design demonstrations based on the information provided in the rubric. 

Methodology 

In seeking to investigate the faculties’ practices in relation to the use of rubrics in teaching, learning 
and assessment, action research was the methodology of choice because of its ability to combine 
action with research to inevitably challenge the status quo (Somekh & Zeicher 2009, p. 19). The 
action research methods enabled the researchers to work together to gather data around the issue as 
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well as draw in a group of practitioners to work together to develop knowledge, develop a critical 
awareness of the issues at hand, and refine practice to address the gaps and silences evident in 
assessment processes in order to encourage justice in and equity of engagement with assessment 
practices for all students (Zeni 2001). The cyclical approach to investigation espoused by action 
research allowed for an introductory cycle which focussed on developing understanding and a 
subsequent cycle which promoted change. Robertson notes, ‘There can be action research for 
specific purposes, action research for action purposes and action research communities that can lead 
to emancipatory practices in education’ (2000, p. 324). The aims of this research project aligned 
with the purposes outlined by Robertson (2000). 

The research undertaken in the SEH fulfilled these purposes in the following ways: 

1 . Action research for specific purposes - Identified aspects of assessment practices that were 
not being overtly and openly communicated with students; Understood how students 
engaged with rubrics as an assessment tool; 

2. Action research for action purposes - Considered the use of rubrics as an assessment tool to 
address the gaps, silences and inconsistencies between declared and actual assessment 
processes; and 

3. Action research communities that can lead to emancipatory practices in education - 
Explored the potentialities of using rubrics as a means of leading students towards 
empowered understanding of assessment practices. 

Phase one of the action research involved a working party of three full-time lecturing staff from the 
SEH who worked together to establish the scope of potential action through the undertaking of an 
audit of practice, a review of literature and the development of recommendations that would 
structure further action. The findings of these activities were then presented to the seven full-time 
and two part-time SEH staff members in the form of a discussion paper and a set of nine 
recommendations. Robust discussion regarding these findings was facilitated by a workshop 
session where in-principle agreement to implement the recommendations was reached. 

Phase two of the action research involved seven full-time staff who applied the nine 
recommendations with undergraduate students across Years 1-4 of the preservice Bachelor of 
Education programs at CHC. The part-time staff members were unable to commit to the study at the 
time. 

Data was collected from participating lecturers and students across Years 2-4 by the means of 
email-back questionnaires. This mode of data collection was chosen firstly because of the minimal 
costs involved in production and distribution of the questionnaires, and also because the data would 
be received in a format which did not require transcription. It was also thought that students would 
respond more readily to an electronic format. This proved to be correct with one class sending in 1 1 
responses (out of a total of 13 responses received from the class) within 4 hours of the instrument 
being sent out. 

Eecturers were requested to respond to open-ended questions about the use of rubrics with their 
classes. Six full-time staff members responded to the questionnaire and one full-time staff member 
did not respond. The qualitative data was analysed by the identification and classification of unique 
meaning units leading to emergent categories (Glesne 1999; Johnson & Christensen 2004). 

The students were asked to respond to a yes/no survey, questioning the nature of their engagement 
with the rubrics across six possible processes. The survey was emailed to students by three 
lecturers explaining the purposes for collecting information and promising anonymity to 
participants. If they agreed to being involved in the study they were asked to send their responses to 
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the survey to their leeturer. This data was eollated quantitatively, linked to the students’ results for 
each task and then stripped of all identifying information. Descriptive statistics such as frequency 
distributions and measures of central tendency were then used to identify patterns in the data 
relating to rubric usage and corresponding assessment outcomes (Johnson & Christensen 2004). 

Findings 

During phase one of the project, two activities contributed significantly to the findings reported to 
all full and fractional time lecturing staff through the ‘discussion paper’ and workshop session. 
These were the audit of practice and the literature review. 

The audit of practice involved the collection of rubrics which were already being used by full-time 
lecturing staff in the SEH. This revealed that two significantly different models for the specification 
of criteria were being used in the SEH. While some units had rubrics based on a ‘curricular 
intentions’ model using the ‘Exit Criteria’ as identified in the accredited unit outline as the criteria 
for the rubric; other units had rubrics based on a ‘task-specific’ model where the nature of the tasks 
and the ‘Exit Criteria’ are mutually unpacked into a set of ‘enabling criteria’ specific to each 
assessment task. The first model results in a single rubric for use with all assessment tasks in a unit, 
and the second results in a unique rubric for each task. Variations were also evident in; the 
specificity of the criteria, the labelling of the standards, the development of the language of the 
standards descriptors, the number of standards considered to be passing grades, the conversion of 
assessment grades to overall grades for a unit, formatting and page orientation. 

The review of literature around the use and construction of rubrics elicited information on different 
types of rubrics and their use both as tools for assessment and teaching. The following model was 
developed to describe and differentiate between different types of rubrics. 







Mid-Range Descriptors 


Benchmark Descriptors 


Curricula 

r 

Intention 

s 


Criteria: ‘unit outcomes’ or ‘exit 
criteria’ 

Qualities: one for each standard 


Criteria: ‘unit outcomes’ or ‘exit 
criteria’ 

Qualities: benchmarks established on 
continuum 


Task- 

Specific 


Criteria: relate to the task 
Qualities: one for each standard 


Criteria: relate to the task 

Qualities: benchmarks established on 

continuum 



Figure 1. Differentiating models of rubrics. 



The information gathered through the audit of practice and literature review contributed to the 
development of nine recommendations about the use of rubrics in the SEH. The scope and nature 
of the recommendations were designed to facilitate the development of comparable practices 
between lecturing staff in relation to the use and development of rubrics for teaching and 
assessment. The purpose of the recommendations was to promote and facilitate predictable, 
comprehensible and equitable communication with all students across all units. After considerable 
discussion and negotiation, the recommendations were articulated as follows: 

1 . Common language will be used when labelling and describing rubrics. 

2. Rubrics will be developed for all assessment tasks. 

3. All rubrics will be overtly linked to the unit outline. 

4. All rubrics will have four passing standards to line up with the four passing exit grades. 

5. Common verbal standard names will be used to label the five standards and these should not 
be abbreviated in the reporting of overall results on a task. The standards shall be: 
Outstanding, High Quality, Commendable, Satisfactory and Unsatisfactory. 

6. All rubrics will include standard descriptors that describe mid-range expectations. 
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7. All criteria on a rubric will be assumed to be equally weighted unless otherwise indicated 
and approved. 

8. Qualitative judgements based on the assigned weightings and the evidence provided in 
students’ performanees, rather than quantitative caleulations, will be used in the alloeation 
of overall results for an assessment task. 

9. All rubries will be tabulated with the deseription of the highest performance on the left and a 
clear demarcation between the passing and failing standards descriptors. 

At the eonelusion of phase one of the project, the negotiated recommendations were approved by all 
staff present at the workshop and agreement was reached about the continuation of the project into a 
second phase. All full-time staff agreed to implement the recommendations and provide data 
relating to this implementation at the conclusion of the following semester. However, the data 
colleeted from lecturing staff paints a picture of variable engagement with the nine 
rec ommendations . 

Four of the six leeturers who submitted the questionnaire reported implementing the full nine 
recommendations in at least one unit in the semester, while the other two listed aspeets they found 
challenging or had ‘forgotten’ to implement. Having said that, all of the lecturers involved were 
positive in their attitude towards the potential of rubrics to make a difference in the assessment 
beliefs and praetiees of students. One lecturer stated that, ‘the rubric has facilitated the level at 
which these tasks should be engaged’. Another reported that she felt that the students had a 
greater understanding of what was expected in the assessment tasks as a result of using the 
rubrics 

All of the leeturers involved in the projeet also reported a wide range of teaehing and learning tasks 
and aetivities that they had undertaken with students in relation to the rubries that they had 
developed. The range of activities undertaken were categorised into three broad actions; namely 
construeting rubries, unpaeking rubries, and providing feedback to students. This showed that the 
lecturers were using the rubrics for both instructional and scoring purposes, though there was 
variation in the depth and level of engagement in the unpaeking proeesses. While some leeturers, 
‘referred to the task specific rubric criteria when “unpacking” the assessment task’, others aetually 
undertook, ‘tutorial activities unpacking them ’ with the students. 

There were also a number of challenges that the leeturers deseribed in relation to the development 
and use of the rubries with students. Most of these were eonneeted to issues of alignment and 
linguistic clarity. Alignment was shown to be useful in facilitating the coverage of unit 
requirements, guiding teaehing and learning and developing evidenee trails from exit eriteria to 
assessment tasks. The lecturers elearly deseribed the significance of the proeess of aligning the 
rubric criteria with the eontent and requirements of the unit. This alignment was deseribed as an 
‘essential tool for effectively evidencing learning with this unit ’. 

This alignment, however, created significant challenges for leeturers; particularly in relation to 
linguistie clarity. Linguistic clarity is foundational to the usefulness of a rubric to the learners; the 
language used in creating the rubric must be clear to the learner and build appropriate bridges 
between the learner and the aeademie discourse relevant to the task. In the development and use of 
rubrics, the lecturers described the process of ensuring linguistic clarity as the most challenging 
eomponent of the process. One lecturer noted that their challenge lay in developing, ‘sub-criteria 
and unpacking of a given criteria in a manner that enhances evidence of quality learning 
outcomes’. While another more specifically described the challenge of, ‘trying to create 
descriptors for each criterion within each standard which aren ’t simply based on semantics (i.e. the 
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changing of one or two adjectives) but truly reflect the changing nature of each (decreasing) 
standard’. 

Achieving linguistic clarity in the development of a rubric is foundational to facilitating student 
engagement and success with them. As one student noted, in an additional comment submitted with 
the survey, ‘The reason I don 't is that in the early days when looking at rubrics, I found them so 
unclear, it only confused the question or task for me It was therefore important to collect data 
from students in relation to units were the lecturing staff felt confident about the quality of the 
rubrics and their adherence to the recommendations. The three units selected involved the four 
lecturing staff who reported that they had implemented all of the recommendations. 

The students were asked to identify which, if any, of six processes they had undertaken as they 
prepared their assessment demonstrations for the various tasks in the selected units. The six 
processes were; 

• Reading the rubric prior to starting, 

• Analysing the rubric prior to starting, 

• Talking about the rubric with other students, 

• Questioning the lecturer about the rubric, 

• Reviewing the rubric while completing the task, and 

• Self-assessing the task against the rubric in order to make improvements. 

The number of these processes that each student undertook, for each assignment they reported on, 
was then counted. This showed differential levels of engagement with the rubrics across the 
surveyed student body, and a fairly even spread with almost as many students using all six 
processes as there was using none of them. Table 1 shows the spread of student engagement with 
the processes as a percentage of the responses received. 



Table 1. Rubric usage as reported by students. 



The number of processes reported: 


0 


1 


2 


3 


4 


5 


6 


Percentage of students reporting this level of 
engagement: 


12% 


13% 


18% 


18% 


16% 


14% 


9% 



This data about students’ rubric usage was then correlated to the results achieved on each 
assessment task. From this analysis, an interesting pattern emerged in relation to the level of 
engagement with the rubrics for a particular task and the results achieved on that task. It was 
observed that rubric usage declined as achievement declined, but, a failure to engage with the 
rubrics did not preclude a high level of achievement. Table 2 illustrates this point by comparing 
the reported engagement with the rubrics for those students who received an ‘Unsatisfactory’ result 
with the reported engagement of those who received an ‘Outstanding’ result for the relevant task. 



Table 2. Assessment achievement compared to the level of reported rubric usage. 



The number of processes reported: 


0 


1 


2 


3 


4 


5 


6 


% of students who received ‘Unsatisfactory’ 
- equivalent to a Fail 


75% 


25% 


- 


- 


- 


- 


- 


% of students who received ‘Outstanding’ 


6% 


6% 


22% 


6% 


17% 


28% 


17% 


- equivalent to a High Distinction 



When this data was then analysed for all levels of attainment to find the mean level of rubric usage, 
the pattern of decline became clearer. Figure 2 describes this decline in rubric usage as the level of 
achievement declines. 
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<u 4.00 
I 3.50 
a 3.00 

1 2.50 

5 2.00 
6 1.50 

<u 1.00 

2 0.50 

I 0.00 

Seriesl 

Figure 2. Mean rubric usage against level of attainment. 

Having identified these two patterns, questions surfaeed about a range of variables that might be 
influencing the data. Of particular concern was the potential for the skills and abilities of individual 
students to influence these results. In order to remove the influence of this variable, it was necessary 
to look at the data about the two different assessment tasks that had been collected from each 
individual student. When this was done, it was noted that while more students maintained the same 
level of rubric usage for both tasks, around 44% of students reported different levels of engagement 
with the rubrics for the different assignments. When the data was rearranged to separate these two 
groups of students, those for who had maintained the level of rubric usage and those who had made 
changes, some other interesting pattern was observed. Firstly, of the students who had made 
changes to their engagement with the rubrics, about half of them (48%) had increased their usage 
for the second task while the other half had decreased their usage. This effectively meant that the 
influence of variables such as the complexity of the task and the timing in the semester had been 
moderated also. Secondly, a correlation between a higher level of engagement with the rubrics and 
a higher level of achievement was observed. In contrast, the results of those students who 
maintained the same level of rubric usage were split fairly evenly between improving, maintaining 
and declining levels of attainment. Figure 3 shows proportionally how the results of students 
improved, were maintained or declined when the student increased their level of engagement with 
the relevant rubric. 
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Figure 3. Change in attainment after increasing rubric usage. 



Conclusions and Recommendations 

As a result of engaging in the process of analysing the data collected from both staff and students, 
three pivotal outcomes emerged. These related to the development of staff competencies in 
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developing and using rubries; the presenee, broadly speaking, of three distinet groups of students in 
terms of their rubrie usage; and the students’ development towards understanding, using and 
developing rubries. 

Firstly, the differential engagement of leeturers with the development and use of rubries in line with 
the nine recommendations was dependent on individual understandings, available time and 
confidence. The lecturers who had been involved in the development of the first phase of the 
project were able to engage in the second phase in a much more consistent fashion and were able to 
facilitate the collection of data from students. However, the staff who were introduced to the 
project and the recommendations through the workshop at the conclusion of the first phase, 
contributed significantly to understanding the processes involved in lecturers engaging with rubrics. 
As with many innovations, the development of lecturers’ competencies in the development and use 
of rubrics was shown to be incremental. In the context of the second phase of this project, three key 
features of this development emerged; these were the lecturers’ competencies in creating, teaching 
about and teaching with rubrics. Professional discussions of and development in all three of these 
aspects of lecturers’ use of rubrics were shown to be needed if the benefits of rubrics as both 
scoring and instructional tools were going to be attained. Furthermore, it was also shown that the 
issue of linguistic clarity was foundational to success in this development. The lecturer’s ability to 
clearly describe the requirements of the task in the language of both the criteria and standards was 
shown to be foundational to the usefulness of the rubrics in the process of scaffolding teaching, 
learning and assessment. Figure 4 is a model that has been developed out of the data to describe the 
process of linguistic clarity that is required to maximise engagement with the relevant academic 
discourses. 

evidence i 



i 



Levels of 


Linguistic 


Language 


, Attainment 


Clarity 


Used ' 



T 



description j 

Figure 4. Linguistic clarity. 

Secondly, throughout both the initial discussions that prompted this research and the data that was 
gathered from staff in phase two of the project there was an assumption evident in the ways that 
student potential and performance was categorised. Overwhelmingly, lecturing staff talked about 
two categories of student; those who were ‘mystified’ or ‘failed to appreciate assessment practices 
and processes’ and those who ‘understand the process from exit criteria to task’. However, the 
data that was gathered from the students did not support this assumption. Rather, the data shows 
that there are in fact three broad categories of students; namely, the mystified, the intuitive and the 
empowered students. The mystified students are those who are unaware of the processes of 
assessment practices in relation to the use of rubrics; these are the students who did not engage with 
the rubrics and consequently did not successfully complete the assessment tasks. The intuitive 
students, on the other hand, were able to successfully complete the assessment task, but not because 
of their ability to engage with the rubric; somehow these students are able to identify the key 
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requirements of a task from the deseription of the task alone. The empowered students are those 
students who are able to suoeessfully eomplete the assessment tasks and are able to use the rubries 
as a tool to assist them in ensuring that the task is fulfilling the requirements. 

Thirdly, in the eontext of a teaeher edueation eourse, the proeess of developing students’ awareness 
and understanding of the assessment praetiees and prineiples that underpin the development and use 
of rubries was highlighted. Just as the leeturing staff were shown to be on a professional journey in 
relation to rubries, so too were the students. From investigating the praetiees of students aeross 
Years 2-4 it was shown that the benefit for the student began with their own learning but would 
infiuenee their teaehing also. 

Having eompleted the seeond phase, it was important to bring these outeomes to bear on the 
original purposes of the projeet and to eonsider the potential impaet that this might have on students 
like Ruth. In doing so, it is apparent that the aims of the projeet have been served by the first two 
phases. Firstly, the audit of praetiee has identified a number of areas of eoneem in terms of the 
eommunieation of assessment proeesses and praetiees and the development of the nine 
reeommendations has led to more open and eonsistent eommunieation aeross the leeturing staff 
involved. Seeondly, a deeper and more diverse pieture of student engagement with assessment 
tasks and the potential of the rubries as a means of empowering students has emerged. Finally, 
signifieant understandings of the proeesses of developing and using rubries as seoring and 
instruetional tools designed to seaffold engagement in relevant aeademie diseourse have developed 
(Hyland 2009). 

These understandings have strengthened the argument for eontinued aetion and researeh in this area. 
Some of the unanswered questions worthy of further investigation inelude; 

• How do the ‘empowered’ students use rubries to help in the assessment proeess? 

• How do the ‘intuitive’ students know what to do when given an assignment? 

• What things are preventing the ‘mystified’ students from engaging with the rubries? 

• What proeesses faeilitate linguistie elarity in the development and use of rubries? 

These questions have beeome the foeus of the third phase in this projeet and have strengthened the 
foeus of the researeh on the potential of rubries to faeilitate just assessment praetiees that are within 
the reaeh of all students. 

But, what about Ruth? Ruth did not eontribute to the student data. She had failed at least one of the 
relevant assessment tasks and mentioned in passing that she didn’t know what to do with the rubries 
so hadn’t answered the questionnaire. Ruth was then invited to partieipate in an individual study 
session to diseuss the proeesses of assessment and how to unpaek a rubrie as a means of 
understanding an assessment task. Ruth aeeepted the offer, engaged in the intervention and has 
applied her new learnings to subsequent assessment tasks. This semester Ruth is eonsistently 
performing at a ‘Credit’ level or better and is on traek to reeord her best grade point average for a 
semester’s work. 
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